A simple duality proof in convex quadratic programming with a quadratic constraint, and some applications

In this paper a simple derivation of duality is presented for convex quadratic programs with a convex quadratic constraint. This problem arises in a number of applications including trust region subproblems of nonlinear programming, regularized solution of ill-posed least squares problems, and ridge regression problems in statistical analysis. In general, the dual problem is a concave maximization problem with a linear equality constraint. We apply the duality result to: (1) the trust region subproblem, (2) the smoothing of empirical functions, and (3) to piecewise quadratic trust region subproblems arising in nonlinear robust Huber M-estimation problems in statistics. The results are obtained from a straightforward application of Lagrange duality. Ó 2000 Elsevier Science B.V. All rights reserved. Keywords: Lagrange duality; Convex quadratic programming with a convex quadratic constraint; Ill-posed least squares problems; Trust region subproblems 1. Convex quadratic programs with an ellipsoidal constraint Consider the problem (P) min y ÿ dy ‡ 1 2 yQy subject to yPy6 d; where Q is a symmetric, positive semide®nite n n matrix, d an n vector not identically zero, P an n n symmetric positive semide®nite matrix, y an n vector, and d a positive scalar. This problem arises in many applications including trust region subproblems of nonlinear programming [9,4], and regularization of ill-posed least squares problems [7]. It is also related to the technique of ridge regression in statistical estimation [7]. Recently, the problem has received renewed interest due to its relation to semide®nite programming; see Ref. [15]. The last reference derives a semide®nite dual problem to (P) for the case where Q is a symmetric, possibly inde®nite matrix. The dual problem derived in Ref. [15] has a single variable and also applies to the convex case while it involves the pseudo-inverse of a certain symmetric matrix. It is a maximization problem over a positive semide®niteness constraint on the matrix Qÿ kI where k is European Journal of Operational Research 124 (2000) 151±158 www.elsevier.com/locate/dsw * Tel.: +90-312-290-1514; fax: +90-312-266-4126. E-mail address: [email protected] (M.Cß . Põnar). 0377-2217/00/$ see front matter Ó 2000 Elsevier Science B.V. All rights reserved. PII: S 0 3 7 7 2 2 1 7 ( 9 9 ) 0 0 1 7 3 3 a scalar. Then, a semide®nite dual to this problem is given, and this primal±dual pair is used to motivate an algorithm for the trust region problem. Other related references that deal with the nonconvex case include Refs. [6,2] where dual problems to the nonconvex quadratic program with an ellipsoidal (or, spherical) constraint are derived. In particular, in Ref. [2] the problem is shown to be equivalent to a convex program through duality. Our purposes in the present note are more modest. We wish to provide the interested reader with a compact and accessible reference on duality pertinent to convex quadratic programs with a single quadratic constraint. We also present a catalogue of three applications from the literature including the trust region subproblems. It is hoped that the present paper will serve to generate more insight to the designers of algorithms for the aforementioned problem class. Although the optimality conditions for the trust region subproblem (with P ˆ I) (or, the regularization of ill-posed least squares problems) are well studied, resulting in ecient algorithms [9,4,7], to the best of our knowledge, derivation of duality for the convex trust region problem has not been exposed before in the simple form given below. In the present note we derive a dual problem to (P) using Lagrange duality [14]. Our dual problem is a concave maximization problem over linear constraints. In particular, in all cases the dual simpli®es to a concave maximization problem with a quadratic term and a nondi€erentiable two-norm term in the objective function. Our approach is essentially inspired from Ref. [17] where a Lagrange dual for entropy minimization problems is given. The main duality result of the present paper can be seen to be similar to the results of Refs. [11±13]. However, we use a more direct and simpler derivation technique from Lagrange duality. Baron [1] derives a Wolfe dual for the problem, which contains a large number of variables despite the simplicity of the derivation. Lagrange duality for such problems is also discussed in Ref. [18] using the theory of `p programming. This last reference discusses weak and strong duality, and uniqueness of solutions as well as regularity of `p programming problems. It is shown that these problems are solvable in polynomial time in Ref. [3]. A specialized interior-point method applied to truss topology design problems was implemented with success in Ref. [10]. In Section 2.1 we apply our duality result to quadratic trust region subproblems of nonlinear programming. In Section 2.2 we discuss the smoothing of empirical functions [19] by quadratic programming. Another contribution of the paper is to show in Section 2.3 that our derivation technique is also extended easily to minimization of piecewise quadratic objective functions over a quadratic (ellipsoidal) constraint. We illustrate this on an important problem from robust statistics. The main result of the paper can be summarized in the following. Proposition 1. (1) The Lagrange dual of (P) is the following concave program (D) max x2Rm; z2Rm; l2R /1…x; l† ÿ 1 2 zTzÿ ld subject to ATz‡ /2…x; l† ˆ d; lP 0; where Q ˆ AA and P ˆ EE. /1…x; l† ˆ ÿ…1=4l†xTx if l > 0; 0 if l ˆ 0; ( /2…x; l† ˆ Ex if l > 0; 0 if l ˆ 0; ( under the condition that l ˆ 0 implies x ˆ 0. (2) The optimal solution of the dual problem …z ; x ; l † for l > 0 and a primal optimal solution y are related by the identities